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Abstract: New continuous and stochastic extensions of 
the minority game, devised as a fundamental model for a mar- 
ket of competitive agents, are introduced and studied in the 
context of statistical physics. The new formulation repro- 
duces the key features of the original model, without the need 
for some of its special assumptions and, most importantly, it 
demonstrates the crucial role of stochastic decision-making. 
Furthermore, this formulation provides the exact but novel 
non-linear equations for the dynamics of the system. 

There is currently much interest in the statistical 
physics of non-equilibrium frustrated and disordered 
many-body systems [jlj. Even relatively simple micro- 
scopic dynamical equations have been shown to lead to 
complex co-operative behaviour. Although several of the 
interesting examples are in areas traditionally viewed as 
physics, it is growingly apparent that many further chal- 
lenges for statistical physics have their origins in other 
fields like biology Q and economics Q. In this letter 
we discuss a simple model whose origin lies in a market 
scenario and show that not only does it exhibit inter- 
esting behaviour in its own right but also it yields an 
intriguingly unusual type of stochastic micro-dynamics 
of potentially more general interest. 

The model we will introduce is based on the the minor- 
ity game (MG) [Q, which is a simple and intuitive model 
for the behaviour of a group of agents subject to the eco- 
nomic law of supply and demand, which ensures that in 
a market the profitable group of buyers or sellers of a 
commodity is the minority one From the perspec- 
tive of statistical physics, these problems are novel ex- 
amples of frustrated and disordered many-body systems. 
Agents do not interact directly but with their collective 
action determine a 'price' which in turn affects their fu- 
ture behaviour, so that minority reward implies frustra- 
tion. Quenched disorder enters in that different agents 
respond in different ways to the same stimuli. There 
are effective random interactions between agents via the 
common stimuli and the cooperative behaviour is remi- 
niscent of that of spin-glasses || , but there are important 
conceptual and technical differences compared with the 
problems of conventional statistical physics. 

The setup of the MG in the original formulation of 0] is 



* E-mail: a.cavagnal@physics.ox.ac.uk 
t E- mail : j . garrahan 1 ©physics . ox . ac . uk 
"'"E-mail: i.giardinal@physics.ox.ac.uk 
§ E-mail: d.sherringtonl@physics.ox.ac.uk 



the following: N agents choose at each time step whether 
to 'buy' (0) or 'sell' (1). Those agents who have made the 
minority choice win, the others lose. In order to decide 
what to do agents use strategies, which prescribe an ac- 
tion given the set of winning outcomes in the last m time 
steps. At the beginning of the game each agent draws s 
strategies randomly and keep them forever. As they play, 
the agents give points to all their strategies according to 
their potential success in the past, and at each time step 
they employ their currently most successful one (i.e. the 
one with the highest number of points). 

The most interesting macroscopic observable in the 
MG is the fluctuation tr of the excess of buyers to sell- 
ers. This quantity is equivalent to the price volatility 
in a financial context and it is a measure of the global 
waste of resources by the community of the agents. We 
therefore want a to be as low as possible. An important 
feature of the MG, observed in simulations 0, is that 
there is a regime of the parameters where a is smaller 
than the value a r which corresponds to the case where 
each agent is buying or selling randomly. Previous stud- 
ies have considered this feature from a geometrical and 
phenomenological point of view Our aim, however, 
is to enable a full analytic solution. 

One of the major obstacles to an analytic study of 
the MG in its original formulation is the presence of an 
explicit time feedback via the memory m. Indeed, when 
the information processed at each time step by the agents 
is the true history, that is the result of the choices of the 
agents in the m previous steps, the dynamical evolution 
of the system is non-Markovian and an analytic approach 
to the problem is very difficult. 

A step forward in the simplification of the model has 
been made in where it has been shown that the ex- 
plicit memory of the agents is actually irrelevant for the 
global behaviour of the system: when the information 
processed by the agents at each time step is just invented 
randomly, having nothing to do with the true time se- 
ries, the relevant macroscopic observables do not change. 
The significance of this result is the following: the crucial 
ingredient for the volatility to be reduced below the ran- 
dom value appears to be that the agents must all react 
to the same piece of information, irrespective of whether 
this information is true or false ]lO| . This result has 
an important technical consequence, since the explicit 
time feedback introduced by the memory disappears: the 
agents respond now to an instantaneous random piece of 
information, i.e. a noise, so that the process has become 
stochastic and Markovian. 
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The model can be usefully simplified even further and 
at the same time generalized and made more realistic. 
Let us first consider the binary nature of the original 
MG. It is clear that from a simulational point of view 
a binary setup offers advantages of computational effi- 
ciency, but unfortunately it is less ideally suited for an 
analytic approach jll|] . More specifically, if we are inter- 
ested in the analysis of time evolution, integer variables 
are usually harder to handle. Moreover, the geometrical 
considerations that have been made on a hypercube of 
strategies of dimension 2 m for the binary setup S, be- 
come more natural and general if the strategy space is 
continuous. Finally, in the original binary formulation of 
the MG there is no possibility for the agents to fine tune 
their bids: each agent can choose to buy or sell, but they 
cannot choose by how much. As a consequence, also the 
win or loss of the agents is not related to the consistency 
of their bids. This is another unrealistic feature of the 
model, which can be improved. For all these reasons, we 
shall now introduce a continuous formulation of the MG. 

Let us define a strategy flasa vector in the real space 
R D , subject to the constraint, \\R\\ = \f~D. In this way the 
space of strategies T is just a sphere and strategies can be 
thought as points on it. The next ingredient we need is 
the information processed by strategies. To this aim we 
introduce a random noise ff(t), defined as a unit-length 
vector in R- , which is (5-correlated in time and uniformly 
distributed on the unit sphere. Finally, we define the 
response b(R) of a strategy R to the information ff(t), as 
the projection of the strategy on the information itself, 

b(R) = R-ff(t) ■ (1) 

This response is nothing else than the bid prescribed by 
the particular strategy R. The bid is now a continuous 
quantity, which can be positive (buy) or negative (sell). 

At the beginning of the game each agent draws s strate- 
gies randomly from T, with a flat distribution. All the 
strategies initially have zero points and in operation the 
points are updated in a manner discussed below. At each 
time step the agent uses his/her strategy with the highest 
number of points. The total bid is: 

N N 

A(t) = 5>(t) rfc) , (2) 

i=l i=l 

where R*(t) is the best strategy (that with the highest 
number of points) of agent i at time t. 

We have now to update the points. This is particularly 
simple in the present continuous formulation. Let us in- 
troduce a time dependent function P(R, t) defined on T, 
which represents the points P of strategy R at time t. 
We can write a very simple and intuitive time evolution 
equation for P, 

P(R,t + l) = P(R,t)-A(t)b(R)/N, (3) 



where A(t) is given by eq.(||). A strategy R is thus re- 
warded (penalized) if its bid has an opposite (equal) sign 
to the total bid A(t), as the supply-demand dynamics 
requires. Now the win or the loss is proportional to the 
bid. 
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FIG. 1. Continuous formulation: The scaled variance 
o j\f~N as a function of the reduced dimension D/N, at s = 2 
and s = 4. The horizontal line is the variance in the random 
case. The total time t and the initial time to are 10000 steps. 
Average over 100 samples, N = 100. 

It is important to check whether the results obtained 
with this continuous formulation of the MG are the same 
as in the original binary model. The main observable of 
interest is the variance (or volatility) a in the fluctuation 
of A, a 2 = limt^oo \ / t * ° + * dt' A(t') 2 . Indeed, we shall 
not consider any quantity related to individual agents. 
We prefer to concentrate on the global behaviour of the 
system, taking more the role of the market regulator than 
that of a trading agent. The main features of the MG are 
reproduced: first, we have checked that the relevant scal- 
ing parameter is the reduced dimension of the strategy 
space d — D/N; second, there is a regime of d where the 
variance a is smaller than the random value ay , showing 
a minimum at d = d c (s), and, moreover, the minimum of 
a(d) is shallower the higher is s ||; see Fig.l. It can be 
shown that all the other standard features of the binary 
model are reproduced in the continuous formulation. 

An interesting observation is that there is no need for 
ff(t) to be random at all. Indeed, the only requirement 
is that it must be ergodic, spanning the whole space T, 
even in a deterministic way. Moreover, if fj(t) visits just 
a sub-space of V of dimension D' < D everything in the 
system proceeds as if the actual dimension was D': the 
effective dimension of the strategy space is fixed by the 
dimension of the space spanned by the information. 

Relations (^) and (||) constitute a closed set of equa- 
tions for the dynamical evolution of P(R, t), whose solu- 
tion, once averaged over if and over the initial distribu- 
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tion of the strategies, gives in principle an exact deter- 
mination of the behaviour of the system. In practice, the 
presence of the 'best-strategy' rule, i.e. the fact that each 
agent uses the strategy with the highest points, makes 
the handling of these equations still difficult. From the 
perspective of statistical physics it is natural to modify 
the deterministic nature of the above procedure by intro- 
ducing a thermal description which progressively allows 
stochastic deviations from the 'best-strategy' rule, as a 
temperature is raised. We shall see that this generaliza- 
tion is also advantageous, both for the performance of 
the system in certain regimes and for the development 
of convenient analytical equations for the dynamics. In 
this context the 'best-strategy' original formulation of the 
MG can be viewed as a zero temperature limit of a more 
general model. 

Hence we introduce the Thermal Minority Game 
(TMG), defined in the following way. We allow each 
agent a certain degree of stochasticity in the choice of the 
strategy to use at any time step. For each agent i the 
probabilities of employing his/her strategy a — 1, . . . , s 
is given by, 



,l3P(Rlt) 



(4) 



6=1 



where P are the points, evolving with eq.Q. The in- 
verse temperature /? = 1 /T is a measure of the power of 
resolution of the agents: when ^-»oo they are perfectly 
able to distinguish which is their best strategy, while for 
decreasing (3 they are more and more confused, until for 
j3 = they choose their strategy completely at random. 
What we have defined is therefore a model which inter- 
polates between the original 'best-strategy' MG (T = 0, 
j3 = oo) and the random case (T — oo, (3 = 0). In the 
language of Game Theory, when T — agents play 'pure' 
strategies, while at T > they play 'mixed' ones [Ti| . 

We now consider the consequences of having intro- 
duced the temperature. Let us fix a value of d belonging 
to the worse-than-random phase of the MG (see Fig.l) 
and see what happens to the variance a when we switch 
on the temperature. We do know that for T = we must 
recover the same value as in the ordinary MG, while for 
T — > oo we must obtain the value o~ r of the random case. 
But in the middle a very interesting thing occurs: <x(T) is 
not a monotonically decreasing function of T, but there 
is a large intermediate temperature regime where a is 
smaller than the random value a r . This behaviour is 
shown in Fig. 2. 

The meaning of this result is the following: even if the 
system is in a MG phase which is worse than random, 
there is a way to significantly decrease the volatility a 
below the random value o~ r by not always using the best 
strategy, but rather allowing a certain degree of individ- 
ual error. 
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FIG. 2. TMG: The scaled variance a/VN as a function of 
the temperature T, at D/N = 0.1, for s — 2. In the inset we 
show a{T)/^/N for D/N = 0.25. 



Note from Fig. 2 that the temperature range where the 
variance is smaller than the random one is more than two 
orders of magnitude large, meaning that almost every 
kind of individual stochasticity of the agents improves 
the global behaviour of the system. Furthermore, as we 
show in the inset of Fig. 2, if we fix d at a value belonging 
to the better-than-random phase, but with d < d c , a 
similar range of temperature still improves the behaviour 
of the system, decreasing the volatility even below the 
MG value. 




FIG. 3. TMG: The scaled variance o/yN as a function of 
the reduced dimension D/N, at different values of the tem- 
perature T = 2 k x 10" 2 , for s = 2. 



These features can be seen also in Fig. 3, where we plot 
a as a function of d at various values of the temperature. 
In addition this figure shows further effects: (i) the im- 
provement due to thermal noise occurs only for d < d c \ 
(ii) there is a cross-over temperature T\ ~ 1, below which 
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temperature has very little effect for d > d c ; (iii) above 
Ti the optimal d c (T) moves continuously towards zero 
and a(d c ) increases; (iv) there is a higher critical tem- 
perature T2 ~ 10 2 at which d c vanishes, and for T > T2 
the volatility becomes monotonically increasing with d. 

We turn now to a more formal description of the TMG. 
Once we have introduced the probabilities nf in eq. (|4|) we 
can write a dynamical equation for them. Indeed, from 
eq.(0), after taking the continuous-time limit, we have, 



n?(t) =-/?<(*) a(t)[R? 



b=l 



^{t)R b A-m, (5) 



where the normalized total bid a(t) is given by, 



A' 



Now fi(t) is a stochastic variable, drawn at each time 
t with the time dependent probabilities set [irj , . . . , 7r|] . 
Note the different notation: Rf are the quenched strate- 
gies, while r~i(t) is the particular strategy drawn at time 
t from the set [R\, . . . , Rf} by agent i with instantaneous 
probabilities [n}(t), . . . , 7if (f)]. In order to better under- 
stand equation (0), we recall that bf(t) = Rf ■ if(t) is the 
bid of strategy Rf at time t (eq.(|l|)) and therefore the 
quantity wf(t) = —B(t)bf(t)ca.n be considered as the 
win of this strategy (cf. eq.(S)). Hence, we can rewrite 
eq. (^J) in the following more intuitive form, 



(7) 



where (w}i = Yll=i n i CO w i(t)- The meaning of equa- 
tion (0) is clear: the probability nf of a particular strate- 
gy Rf increases only if the performance of that strategy 
is better than the instantaneous average performance of 
all the strategies belonging to the same agent i with the 
same actual total bid [[L3| . 

Relations (^J) and (q) are the exact dynamical equa- 
tions for the TMG. They do not involve points nor mem- 
ory, but just stochastic noise and quenched disorder, and 
they are local in time. From the perspective of statistical 
mechanics, this is satisfying and encouraging. However, 
these equations differ fundamentally from conventional 
replicator and Langevin dynamics. First, the Markov- 
propagating variables are themselves probabilities. Sec- 
ond, there are two sorts of stochastic noises, as well as 
quenched randomness. Third, and more importantly, the 
stochastic noises enter non-linearly, one independently 
for each agent via probabilistic dependence on the it 
themselves, the other globally and quadratically. They 
thus provide interesting challenges for fundamental trans- 
fer from microscopic to macroscopic dynamics, including 
an identification of the complete set of necessary order 
parameters jl4j . We shall address the problem of finding 
a solution of the TMG equations in a future work. 



Finally, let us note that the TMG (as well as the MG) 
is not only suitable for the description of market dynam- 
ics. Indeed, any natural system where a population of 
individuals must organize itself in order to optimize the 
utilization of some limited resources, is qualitatively well 
described by such a model. We hope that the thermal 
model we have introduced in this Letter will give more 
insight into this kind of natural phenomena. 

We thank L. Billi, P. Gillin and P. Love for many sug- 
gestions, and N.F. Johnson and I. Kogan for useful dis- 
cussions. This work was supported by EPSRC Grant 
GR/M04426 and EC Grant ARG/B7-3011/94/27. 



[1] See, for example, J. -P. Bouchaud, L.F. Cugliandolo, J. 
(6) Kurchan and M. Mezard in Spin Glasses and Random 

Fields, edited by A. P. Young, (World-Scientific, Singa- 
pore, 1998); Nonequilibrium Statistical Mechanics in One 
Dimension, edited by V. Privman, (Cambridge Univer- 
sity Press, 1997). 
[2] See, for example, Physics of Biomaterials, edited by T. 
Riste and D. Sherrington, (Kluwer Academic Press, Dor- 
drecht, 1996); Landscape Paradigms in Physics and Bi- 
ology, edited by H. Frauenfelder et al., (North-Holland, 
Amsterdam, 1997). 
[3] The Economy as an Evolving Complex System, edited by 
P.W. Anderson, K. Arrow and D. Pines (Addison- Wesley, 
Redwood City, 1988). 
[4] D. Challet and Y.-C. Zhang, Physica A 246, 407 (1997). 
[5] See e.g. W.B. Arthur, Amer. Econ. Assoc. Papers and 

Proc. 84, 406 (1994); Science, 284, 107 (1999). 
[6] M. Mezard, G. Parisi and M. A. Virasoro, Spin Glass 
Theory And Beyond (World Scientific, Singapore, 1986). 
[7] R. Savit, R. Manuca and R. Riolo, Phys. Rev. Lett. 82, 
2203 (1999). 

[8] D. Challet and Y.-C. Zhang, Physica A 256, 514 (1998); 
Y.-C. Zhang, Europhys. News 29, 51 (1998); N.F. John- 
son, M. Hart and P.M. Hui, Physica A, 269, 1 (1999); R. 
D'hulst and G.J. Rodgers, Physica A, 270, 222 (1999). 
[9] A. Cavagna, Phys. Rev. E, 59, R3783 (1999). 
[10] In economics such a situation is described as sun-spot 
equilibrium; see e.g. M. Woodford, Econometrica, 58, 
277 (1990). (We thank M. Marsili for drawing our atten- 
tion to sun-spots.) 

[ 11] See, however, D. Challet and M. Marsili, pond 

PI 



mat/9904071 



J. Hofbauer and K. Sigmund, Evolutionary Games 
and Population Dynamics, (Cambridge University Press, 
1998). 

[13] Equations ([?]) are not replicator equations jl3] . The latter 
describe deterministic processes, while eqs.(CT correspond 
to a dynamics which involves two sorts of stochastic noise, 
together with quenched disorder. Stochastic learning pro- 
cesses are described by replicator equations only if the 
continuous-time limit is taken in a particular way; see T. 
Borgers and R. Sarin, J. Econ. Theory, 77, 1 (1997). 

[14] A.C.C. Coolen, S.N. Laughton and D. Sherrington, Phys. 
Rev. B 53, 8184 (1996). 



4 



